* Add DeepNestedSequenceProcessor and DeepNestedFloatsProcessor for handling deeply nested sequences

* Add unit tests for DeepNestedSequenceProcessor and DeepNestedFloatsProcessor

* Refactor EmbeddingModel to support DeepNestedSequenceProcessor and DeepNestedFloatsProcessor, and support mask outputs

* Refactor AdaCare model to integrate new dataset and processor classes, update input handling, and forward propagation logic

* Remove output mask handling for passthrough tensors in EmbeddingModel forward method

* Add unit tests for AdaCare model including initialization, forward and backward passes, loss checks, and output shapes

* Update mortality prediction example script for MIMIC-III using adacare

* Remove property decorator from size function in StageNetTensorProcessor class

…iuc#643)

* init commit for 1 solution

* also change ex cache

* init commit

* commit new test case and fixes

* new update

* minor update to the number of workers used, turns out it does make a difference in processing speed

* update again

* update with comprehensive icd medcode vocabs

* fix typo in test cases and stagenet to match other processors

* already tested with full training on holdout sets, etc.

* more typo fixes

* split to make sure things are more clear on how to apply and a separate blog post on the highlights for ml4h

…ead and skip around in the call .rst file (sunlabuiuc#657)

…details (sunlabuiuc#661)

breaking rules, but need to just update this.

* Fix SequenceProcessor

* Fix up

* Fix incorrect test

* init commit for now, will merge later after memory fixes

* fixes to bugs

* Polars fix bug of OOM on large table join

* Fix type hint

* Add cache_dir

* Remove to_lower as this is a no-op

* Add caching behaviour

* Add test case

* Add StreamingParquetWriter

* write samples

* Add SampleBuilder

* Fix Mimic4

* fix incorrect dev mode

* change fit to take Iterator

* update test

* rename

* update fit to use Iterable

* Fix SampleBuilder

* Fix tsv test

* Fix base dataset test

* cache processed data

* save schema for SampleBuilder

* Fix sampledataset

* Fix multi-worker crashes

* update test

* Fix non-pickable

* Fix get_dataloader

* Fix embedding

* support split

* Fix test

* Fix collate_fn

* fix conflicting cache dir

* update test

* add create_sample_dataset to convert list of samples to SampleDataset

* test new dataset

* Update docs

* Fix test

* Fix test

* Fix test

* Fix test

* Fix test

* add InMemorySampleDataset

* support InMemorySampleDataset

* Fix test

* Fix tests

* Fix test

* Fix test

* support set_shuffle for InMemorySampleDataset

* Fix in memory dataset subset

* Fix test

* update sample dataset test

* Add deps

* Fix test

* commit for fixing model docs

* fix adacare docstrings

* fix for python 3.10 override typing incompatibility, but still struggling with problems with out of memory?

* organize for benchmarking scripts

* add deps

* add _scan_csv_tsv_gz

* convert load_table to dask

* convert load_data to dask

* convert global_event_df to dask

* Fix base dataset test

* Fix bug

* Fixup

* Fixup

* fixup

* main guard

* fix incorrect null value handling

* change back to ms to mimic old pyhealth beahviour

* add TODO

* main guard check

* fix nullable issue?

* revert change

* update API layers

* Fix cache test

* support remote url

* fix test

* fix incorrect type in nw

* less worker

* non-dev for memtest

* fix incorrect behaviour on notebook & make sure dask excption throw at dask

* update on installation details and recommended settings for use with PyHealth due to new change in backend

* additional clarifications here

---------

Co-authored-by: Yongda Fan <yongdaf2@illinois.edu>

* new alpha ver. with streaming changes

* minor bug fix towards timeseries processor where in python 3.12, naming of class variables and function names conflict when the same string

* fixes for timeseries processor, bugs in datetime parsing, and other details

* update to examples to run these things

* version update again

* doc updates to be more accurate, changed the error raising, and also added another bug fix to where dask leverages its temporary file usage to be in line with the cache_dir

* add num_workers to BaseDataset

* Fix test

* Fix MIMIC4

* use multi-process mode when not in notebook to speed up the process

)

* Fix incorrect null handling for timestamp

* add more test case

* newsletter and doc updates for memory optimizations

* update install version to use latest

* clarify on what task we're talking about

* Update Agent, PyHealth 2.0

* Address Copilot review: fix 4D tensor handling, reward function, and schema

- Add 4D->3D pooling for NestedSequenceProcessor (sum across codes dim)
- Fix binary/multilabel reward to provide feedback for both classes
- Change schema from 'sequence' to 'nested_sequence' throughout
- Update test samples to use nested visit structure

---------

Co-authored-by: Steier <637682@bah.com>

* Update ConCare model for PyHealth 2.0 with testing

* add example MIMIC 4 demo, ConCare

* fixed ipynb, concare

* Address Copilot review feedback: fix imports, formatting, and add tests

---------

Co-authored-by: Steier <637682@bah.com>

* Update Deepr model to PyHealth 2.0

* Address review feedback for Deepr model

- Add integer type check for window parameter
- Improve error messages with shape information
- Add paper reference and usage examples to docstring
- Add nested sequence test case (4D tensor handling)
- Add MIMIC-IV mortality prediction example notebook

---------

Co-authored-by: Steier <637682@bah.com>

* rename to avoid naming collision

* Support multi-worker for task transformation

* Fix import

* Fix deadlock

* better IPC communication

* fix bug when num_workers == 1

* Fix UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown

* Fix crash

* Fix single worker mode.

* Use batch processing

* ensure locality on data access.

* Fix polars threads.

…Health into eeg-gcnn-contribution

…D model (sunlabuiuc#981)

* dl4h final project kobeguo2 - CaliForest

* Update CaliForest to require explicit fit before inference

* Remove unused logit_scale from CaliForest

- Update Robert Coffey's NetID from racoffey2 to rc37
- Rename heading from 'Signal Processing (EEGGCNNDiseaseDetection)' to
  'Signal Processing'; add note clarifying it is implemented in both
  EEGGCNNRawDataset.precompute_features() (batch) and
  EEGGCNNDiseaseDetection.__call__() (streaming)
- Correct PSD band names and ranges to match actual code and FigShare
  arrays: delta/theta/alpha/beta/low_gamma/high_gamma with boundaries
  matching _BAND_RANGES (7.5, 13.0, 30.0, 40.0 Hz)
- Attribute graph adjacency alpha formula to EEGGCNNDataset where it lives
- Fix adjacency description: 'geodesic distance' (not 'arc length on unit sphere')
- Remove non-existent download_lemon.py reference from Quick Start

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Update Robert Coffey's NetID from racoffey2 to rc37
- Rename heading from 'Signal Processing (EEGGCNNDiseaseDetection)' to
  'Signal Processing'; add note clarifying it is implemented in both
  EEGGCNNRawDataset.precompute_features() (batch) and
  EEGGCNNDiseaseDetection.__call__() (streaming)
- Correct PSD band names and ranges to match actual code and FigShare
  arrays: delta/theta/alpha/beta/low_gamma/high_gamma with boundaries
  matching _BAND_RANGES (7.5, 13.0, 30.0, 40.0 Hz)
- Attribute graph adjacency alpha formula to EEGGCNNDataset where it lives
- Fix adjacency description: 'geodesic distance' (not 'arc length on unit sphere')
- Remove non-existent download_lemon.py reference from Quick Start

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

EEG-GCNN: Dataset, Task, Models, and Neurological Disease Detection Pipeline

…gcnn-contribution

…processing section

Explain that all training, holdout, and ablation runs use EEGGCNNDataset
(alpha parameter) + EEGGCNNClassification (excluded_bands) — not
EEGGCNNDiseaseDetection. Document the latter as an available streaming
alternative with configurable adjacency_type and bands, but clarify it
was not exercised in the runs described in this document.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…gcnn-contribution

End-to-end example script following PyHealth naming convention
({dataset}_{task_name}_{model}.py). Demonstrates loading FigShare
pre-computed EEG-GCNN arrays, applying EEGGCNNClassification task,
training EEGGraphConvNet with 10-fold CV and WeightedRandomSampler,
and reporting patient-level AUROC.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

examples/eeg_gcnn/ and examples/eeg_gatcnn/ already contain the full
set of end-to-end scripts from the eeg-gcnn-contribution branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…py removed

Consolidates tests/test_eeg_gcnn.py (EEGGCNNDiseaseDetection, raw path)
into tests/core/test_eeg_gcnn_dataset.py alongside the existing FigShare
path tests. Fixes stale import (eeg_gcnn_nd_detection →
eeg_gcnn_disease_detection) and task_name assertion. Total: 41 tests,
all passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Brings in post-PR-#3 changes from the EEG-GCNN pipeline branch:
- docs: updated eeg_gcnn_pipeline.rst (band names, adjacency attribution,
  EEGGCNNDiseaseDetection vs EEGGCNNRawDataset clarification)
- examples: renamed scripts to required convention
  (eeg_gcnn_classification_gcn_training.py, _evaluation.py, etc.)
- examples: added sample raw + precomputed data for quick-start testing
- tests: consolidated 41-test suite into tests/core/test_eeg_gcnn_dataset.py
  (merged raw-path tests, fixed stale import, removed tests/test_eeg_gcnn.py)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Adds TestRawDatasetPrecompute (12 tests) to test_eeg_gcnn_dataset.py,
exercising EEGGCNNRawDataset.precompute_features() end-to-end against
the real EDF/BrainVision sample files in
examples/eeg_gcnn/sample_raw_data/. Verifies all 5 output files are
created with correct shapes, consistent window counts, valid labels,
and finite PSD values. Total suite: 53 tests, all passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…set.py

Separates tests by data path per Robert's feedback:
- test_eeg_gcnn_dataset.py: reverted to original 18 tests covering the
  FigShare path (EEGGCNNDataset + EEGGCNNClassification)
- test_eeg_gcnn_raw_dataset.py: new file with 35 tests covering the raw
  path (EEGGCNNRawDataset + EEGGCNNDiseaseDetection), including a
  TestRawDatasetPrecompute integration class that runs the full
  preprocessing pipeline against committed EDF/BrainVision sample files

Also removes pyhealth/eeg-gcnn-data/ (untracked local data directory).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Restores the missing section comment block — file is now identical to
the original tests/core/test_eeg_gcnn_dataset.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Superseded by tests/core/test_eeg_gcnn_raw_dataset.py which covers all
23 original tests plus 12 new precompute_features integration tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Band table was wrong — corrected to match _BAND_RANGES in eeg_gcnn_raw.py:
  beta:       13.0–30.0 → 13.0–16.0
  low_gamma:  30.0–40.0 → 16.0–30.0
  high_gamma: 40.0–50.0 → 30.0–40.0

Also removed reference to download_lemon.py which does not exist in the repo.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Local data directory should not be tracked in version control.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Superseded by tests/core/test_eeg_gcnn_raw_dataset.py which covers all
original tests plus 12 precompute_features integration tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…est documents.